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^ I Abstract. - Multi-agent complex systems comprising populations of decision-making parti- 

rj: . cles, have many potential applications across the biological, informational and social sciences. 

• ' We show that the time-averaged dynamics in such systems bear a striking resemblance to 

fj , conventional many-body physics. For the specific example of the Minority Game, this anal- 

• 1— ( . ogy enables us to obtain analytic expressions which are in excellent agreement with numerical 

K^_^' simulations. 

Multi-agent simulations are currently being used to study the dynamical behavior within 
a wide variety of Complex Systems [1]. Within these simulations, N decision-making particles 
or agents (e.g. commuters, traders, computer programs [2-4]) repeatedly compete with each 
Cn , other for some limited global resource (e.g. road space, best buy/sell price, processing time) 
I ' using sets of rules which may differ between agents and may change in time. The population 
r-^ , is therefore competitive, heterogeneous and adaptive. A simple version of such a scenario, 
\^ ' which has generated more than one hundred papers since 1998, is the Minority Game (MG) 
O '. of Challet and Zhang [2-9]. 

(/5 ' Here we show that the time-averaged dynamics of such multi-agent systems - in particular, 

O , their n-point correlation functions - can be interpreted as a generalization of conventional 
c/3 ' many-body physics [10]. We also show that these correlation functions can be evaluated 
^~>, accurately if one regroups the agents into clusters of like particles (i.e. crowds) and their anti- 
correlated mirror-images (i.e. anticrowds). When applied to the MG, this approach yields a 
set of analytic results which are in excellent agreement with the numerical findings of Savit et 
al. (see Fig. 1) [6]. Although there have been many other MG theories proposed to date [5], 
none of these provides an analytic description of this Savit-curve [6] (Fig. 1) over the entire 
}J^ • parameter space. 

5r ! Figure 2 shows a generic setup of such a multi-agent system. The Minority Game represents 

a specific example of this setup, in which the identity of the minority group provides the global 
outcome - however, what follows does not depend specifically on such a minority rule. There 
are N agents (e.g. commuters) who repeatedly decide between two actions at each timestep t 
(e.g. +1/ — 1 = take route A/B) using their individual S strategies. The agents have access to 
some common information fj,{t). For example, Fig. 2 shows /x(i) to be the past few outcomes 
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1 - Results for the standard deviation of fluctuations D^ in the Minority Game. Numerical 
results correspond to 20 different runs at each A'^ and -Rmax- The theoretical curves are generated 
using the analytic expressions in the text. The shaded area bounded by the upper and lower curves 
shows our theoretical prediction of the numerical spread for a given N . In line with the original 
numerical results of Ref. [6] , we have chosen successive -Rmax tick- values to increase by a factor of 4. 
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Fig. 2 - General setup: At timestep t, each agent decides between action —1 and action +1 based on 
the predictions of the S strategies that he possesses. Ai'_i[t] agents choose —1, and Af-|-i[t] choose +1. 
A global outcome or 1 is assigned depending on the rules of the game (e.g. minority group wins). 
Strategies are rewarded/penalized one virtual point according to whether their predicted action would 
have been a winning/losing action. 
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Fig. 3 - Left: strategy space for m — 2, together with some example strategies. The reduced strategy 
space contains 2.2™ = 2P strategies with specific correlations (i.e. fully correlated, uncorrelated or 
anti-correlated [4,5]). Right: strategy allocation tensor ^ in the case oi S = 2 strategies per agent. 
Each square shows the number of agents who were assigned strategy R and then R' . 



of the system where or 1 could represent route A or B being the least crowded, respectively. 
Hence /z(i) is, in the setup of Fig. 2, a binary string of outcomes. Each strategy, labelled R, 
comprises a particular action a'^ ' = ±1 for each /z(i) £ {fi{t)}, and the set of all possible 
strategies constitutes a strategy space 9 = {R}- Figure 3 (left) gives an example of such a 
strategy space in the case that each agent has information fi{t) corresponding to the most 
recent m = 2 outcomes. Each agent has some subset S of these strategies, hence the strategy 
allocation among agents can be described in terms of a rank-S* tensor ^ [9] where each entry 
gives the number of agents holding a particular combination of S strategies. Figure 3 (right) 
gives an example of this strategy allocation matrix ^ for 5 = 2. We assume VP to be constant 
over the timescale for which time-averages are taken. A single VP 'macrostate' corresponds 
to many possible 'microstates' describing the specific partitions of strategies among the N 
agents [9]. To allow for large strategy spaces and large sets of global information, we can 
convert the strategies {R} to their decimal equivalents and label them from R ^ 1 ^ Rmax, 
where i?max = 2.2™, in order of increasing magnitude. Hence for the example strategies in 
Fig. 3, we can relabel strategy [—1 — 1 — 1 — 1] as i? = 1, [—1 — 1 +1 -|-l]asi? = 4 and 
[-1-1 +1 +1 -|-l]asi?=: i?i„ax — 8. Hence R mimics a 'coordinate' in that if an agent is using 
a particular strategy R, the agent can be thought of as sitting on a line at coordinate R. For 
large strategy spaces, there will be many possible i?-values - for example, m > 3 means that 
there are 2.2™ > 16 strategies in the reduced strategy space of Fig. 3. The line of ordered 
strategies, and hence allowed i?-values, will therefore appear quite dense. Given this, we will 
take the liberty of writing down sums over R as integrals over R in the rest of this paper. 

We denote the number of agents choosing —1 (e.g. take route A, or buy a given stock) 
and -|-1 (e.g. take route B, or sell a given stock) as N_i{t) and N^i{t) respectively. Many 
macroscopic quantities of interest can be written in terms of N_i{t) and N^i{t). For example, 
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the excess trafBc on route A as compared to route B, or the excess number of buyers over sellers 
of a given stock and hence its price-change at time t, can be written as D{t) — N^i(t) — N^i{t). 
Hence we will focus here on D{t), which is given by: 

D{t)= f ^^^'^^dRaf^nf^-'", (1) 

where nj^ is the number of agents using strategy R at time t, and S_{t) is the current score- 
vector denoting the past performance of each strategy [9]. In particular, the element Suit) 
represents the net number of successful predictions of strategy R up to time t. Therefore, 
the combination of S_(t), ^ and the game rule (e.g. use strategy with best or second-best 

performance to date) defines n"^ . The action a^ — ±1 is determined uniquely by n{t). 
In conventional many-body physics, we are typically interested in the following statisti- 
cal properties of macroscopic measurable quantities such as D{t): (i) the moments of the 
probability distribution function (PDF) of D{t) (e.g. mean, variance, kurtosis) and (ii) the 
correlation functions that are products of D{t) at different times ti ~ t, t2 — t + t, t^ — t + t' 
etc. (e.g. autocorrelation). Numerical multi-agent simulations typically average over time 
t and then over configurations {^}. A general expression to generate all such functions, is 
therefore 

jj(r,r',r",...) ^ (^^D{t,)D{t,) . . . D{tp)),)^ (2) 

™ dR, dR,... dR, af^^^af^^^ . . . af^^^n^l^^'-^f^^ . . . nf^'^* 

dRr dR2 ...dRp ^<J\/(^'(i?i,i?2,. . .,Rp-M,t2, ■ ■ . ,ip)nf;^'^*nf;^''* . ..nf/^'^ 

where 

V^''HRuR2...,Rp-MM.-.,tp)^4l"^ail'^---4lr^ (3) 

resembles a time-dependent, non-translationally invariant, p-body interaction potential in R = 
(-Ri, i?2 ■ • ■ ! -Rp)-space, between p charge-densities {nj^ } of like-minded agents. Note that 
each charge-density n-^ now possesses internal degrees of freedom determined by S_{t) and 

^. Since {nj^, } are determined by the game's rules, Eq. J^J can be applied to any multi- 
agent game, not just the MG. We focus here on moments of the PDF of D{t) where {ti} = t 
and hence {r} — 0. Discussion of temporal correlation functions such as the autocorrelation 

(t) 

D2 will be reported elsewhere. We consider explicitly the variance D2 to demonstrate the 
approach, noting that higher-order moments such as D^ (i.e. kurtosis) which classify the non- 
Gaussianity of the PDF, can be treated in a similar way. The potential \/(-^) is insensitive to 
the configuration-average over {^}, hence the mean is given by [11]: 

If the game's output is unbiased, the averages yield Di — 0. This condition is not necessary - 
one can simply subtract Df from the right hand side of the expression for D2 below - however 
we will take Di = for clarity. The variance D2 measures the fluctuations of D{t) about its 
average value: 

D2^ f Z'"'"' dRdR' (v^'\R,R';t) (nf'^^*nf^^*\ ) (5) 

J JR,R' = 1 \ \ /*/* 
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where V'^^'>{R,R';t) = af^ o!^i acts like a time-dependent, non-translationally invariant, 
two-body interaction potential in (i?, _R')-space. 

These effective charge-densities and potential fluctuate in time. It is reasonable to assume 
that the charge densities fluctuate around some mean value, hence rr^ rrj^, = nunff + 

^"rW'' (*) "^'^^^ mean nRURi plus a fluctuating term e-^j^,' (t). This is a good approximation 
if we take i? to be a popularity-ranking (i.e. the i?th most popular strategy) or a strategy- 
performance ranking (i.e. the -Rth best-performing strategy) since in these cases nr^ will 
be reasonably constant [4]. For example, taking R a.s a popularity-ranking implies nj^J^ > 
n"^^2 ^ ^R=3 — • ■ -J thereby constraining the magnitude of the fluctuations in the charge- 
density n;^ . Hence 

D2 = J J ™ dRdR' (v^^\R,R';t) {nRUR, + sf^/'' (t)) J ^ . (6) 

We will assume that e-^J/ (t) averages out to zero. In the presence of network connections 
between agents, there can be strong correlations between these noise terms ej^j^,' {t) and 
the time-dependence of V'^^\R,R';t), implying that the averaging over t should be carried 
out timestep-by-timestep [12]. For MG-like games without connections, the agents cannot 
suddenly access larger numbers of strategies and hence these correlations can be ignored. 
This gives 

D2= I I dRdR' (v^^\R,R';t)) urur, . (7) 

As in conventional many-body theory, the expectation value in Eq. (UJ can be 'contracted' 
down by making use of the equal-time correlations between {a^ }. As is well-known for 
MG-like games [2,5,8], we can safely work in the so-called reduced strategy space which is 
constructed such that any pair R and R' are either (i) fully correlated, i.e. a^ — a^, for all 
/i(t); (ii) anti-correlated, i.e. a'^ ' = — a^, ' for all n{t); (iii) uncorrelated, i.e. a'^ ' = a^, ' for 

half of {/^(i)} while a'^ ' = — a^, for the other half of {^(i)}. In other words, one can choose 
two subsets of the strategy space 8, i.e. Q — U (B U, such that the strategies within U are 
uncorrelated, the strategies within U are uncorrelated, the anticorrelated strategy of i? £ [/ 
appears in U, and the anticorrelated strategy of i? e [/ appears in U. We can therefore break 
up the integrals in Eq. {T)): {i) R' = R (i.e. fully-correlated) hence -^ /^'"i" dfia'^a'^, — 1 

and {v'-^^R,R';t))^ = 1; (ii) R' = R (i.e. anticorrelated) which yields ^ /^rf rfA*«' = 
— 1. If all possible global information values {/i} are visited reasonably equally over a long 
time-period, this implies (y^'^\R,R'\t)) = —1. For the MG, for example, {/^} corresponds 
to the m-bit histories which indeed are visited equally for small to. For large to, they are 
not visited equally for a given ^, but are when averaged over all ^. If, by contrast, we 
happened to be considering some general non-MG game where the /x's occur with unequal 
probabilities p^, even after averaging over all ^P, one can simply redefine the strategy subsets 
U and U to yield a generalized scalar product, i.e. — ^^ f^^i" diia'^a'^, p^ = —1 (or in 

case (iii)). (iii) R' L R (i.e. uncorrelated) which yields ^ !^l=T ^M^fl/^fl' ~ ^ ^"^"^ hence 
{V^^\R,R']t))^ = 0. Hence 



D2 = dRdR' {V'^^HR,R';t)) urur, = / dR {hrhr ~ URnj^) 

J JR.R' = 1 ^ ' * Jr=1 
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Jr.i£U 

^ f dR {UR - n^f . (8) 

JReu 

Equation (jHJ must be evaluated together with the condition which guarantees that the total 
number of agents N is conserved: 

N ^ dRnR= dR {ur + n-^) . (9) 

Equation © has a simple interpretation. Since ur and n-^ have opposite sign, they act like two 
charge-densities of opposite charge which tend to cancel each other out: nR represents a Crowd 
of like-minded people, while n-^ corresponds to a like-minded Anticrowd who always do exactly 
the opposite of the Crowd regardless of the specific /i(t). We have effectively renormalized 
the charge-densities nj^ and nj^, and their time- and position-dependent two-body 
interaction V'^'^\R^R']t) = a^ a^, , to give two identical Crowd-Anticrowd 'quasiparticles' 
of charge-density {ur — n-s) which interact via a time- independent and position- independent 



interaction term V^^ = 1. The different types of Crowd-Anticrowd quasiparticle in Eq. © 
do not interact with each other, i.e. {ur — n-n) does not interact with [uri — n-sr) \i R ^ 
R' . Interestingly, this situation could not arise in a conventional physical system containing 
collections of just two types of charge (i.e. positive and negative). 

A given numerical simulation will employ a given strategy-allocation matrix (i.e. a given 
rank- 5* tensor) ^. As i?,nax increases from 1 ^ oo, ^ tends to become increasingly disordered 
(i.e. increasingly non-uniform) [4, 9] since the ratio of the standard deviation to the mean 
number of agents holding a particular set of S strategies is equal to [(i?max ~ 1)/-^]^- There 
are two regimes: (i) A 'high-density' regime where i?max ^ N . Here the charge-densities {ur} 
tend to be large, non-zero values which monotonically decrease with increasing R. Hence 
the set {ur} acts like a smooth function n{R) = {ur}. (ii) A 'low-density' regime where 
-Rmax ^ N ■ Here ^ becomes sparse with each element ^r,r',r",... reduced to or 1. The 
{nR^ should therefore be written as I's or O's in order to retain the discrete nature of the 
agents, and yet also satisfy Eq. [4]. Depending on the particular type of game, moving 
between regimes may or may not produce an observable feature. In the MG, for example, Di 
does not show an observable feature as i?max increases - however D2 does [6] . We leave aside 
the discussion as to whether this constitutes a true phase-transition [5,9] and instead discuss 
the explicit analytic expressions for D2 which result from Eq. ©. It is easy to show that 
the mean number of agents using the Xih most popular strategy (i.e. after averaging over ^) 
is [4]: 



nx ^ N 



1-i^^f-ri- ^ 



(10) 



-^niax / \ ^n 

The increasing non-uniformity in VP as i?max increases, means that the popularity-ranking of 
R becomes increasingly independent of the popularity-ranking of R. Using Eq. (|l()|l with 
S = 2, and averaging over all possible R positions in Eq. ^ to reflect the independence of 
the popularity-rankings for R and R, we obtain: 



D-, = Max 



1 - i?,^;^ , TV 1 

3i?max V / V -Rn 



(11) 



The 'Max' operation ensures that as i?max increases and hence {ur} — > 0,1, Eq. @ is still 
satisfied [4]. Equation ((TT|l underestimates D2 at small i?max (see Fig. 1) since it assumes that 
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the rankings of R and R are unrelated, thereby overestimating the Crowd- Anticrowd cancel- 
lation. By contrast, an overestimate of D2 at small -Rmax can be obtained by considering the 
opposite limit whereby '5 is sufficiently uniform that the popularity and strategy-performance 
rankings are identical. Hence the strategy with popularity-ranking X in Eq. Hl()|l is anticorre- 
lated to the strategy with popularity-ranking Rmax + 1 — X. This leads to a slightly modified 
first expression in Eq. (fTT|l : -^^ — (1 — ^max)- Figure 1 shows that the resulting analytical 

^^'^^ . . . . 1/2 

expressions reproduce the quantitative trends in the standard deviation D2 observed numer- 
ically for all N and i?max7 cind they describe the wide spread in the numerical data observed 
at small i?max- 

An important practical implication of the present paper is that the wide range of cluster- 
based approximation schemes developed in conventional many body-theory, might therefore 
usefully be extended to capture the dominant correlations in multi-agent competitive popula- 
tions. Such generalizations will undoubtedly raise interesting issues for the Physics community, 
concerning the precise manner in which time-averages and configuration-averages should be 
performed within these traditional approximation schemes. 
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